Color contrast enhanced rendering method, device and system suitable for optical see-through head-mounted display

ABSTRACT

The present invention discloses a color contrast enhanced rendering method, device and system suitable for an optical see-through head-mounted display. The method includes: (1) acquiring a background environment in real time to obtain a background video and performing Gaussian blur and visual field correction on the video; (2) converting an original rendering color and a processed video color from an RGB color space to a CIELAB color space scaled to a unit sphere range; (3) finding an optimal rendering color based on the original rendering color and the processed video color in the scaled CIELAB space according to a set color difference constraint, a chromaticity saturation constraint, a brightness constraint and a just noticeable difference constraint; and (4) after converting the optimal rendering color back to the RGB space, performing real-time rendering by using the optimal rendering color of the RGB space. The method is widely applied and capable of obviously improving the discrimination between a virtual content and a background environment and supporting various virtual scenes and background environments without a pre-preparation process.

This is a U.S. national stage application of PCT Application No.PCT/CN2021/070529 under 35 U.S.C. 371, filed Jan. 6, 2021 in Chinese,claiming priority to Chinese Patent Applications No. 202010181920.3,filed Mar. 16, 2020, all of which are hereby incorporated by reference.

TECHNICAL FIELD

The present invention belongs to the field of real-time rendering, andin particular, relates to a color contrast enhanced rendering method,device and system suitable for an optical see-through head-mounteddisplay.

BACKGROUND TECHNOLOGY

With the continuous innovation of optical see-through head-mounteddisplays (hereinafter OST-HMD for short), the related mixed reality(hereinafter MR for short) is also developed continuously. Differentfrom the conventional virtual reality head-mounted display or videosee-through head-mounted display, the OST-HMD has a semitransparentoptical lens, so the OST-HMD may present real and non-drawn ambientenvironments and virtual and drawn display contents at the same time.This design enables a user to perceive the surrounding environmentwithout delay and with high fidelity when wearing the OST-HMD, so thatthe discomfort of the user who wears the video see-through head-mounteddisplay is greatly alleviated, but new problems ensued. The opticalstructure of the OST-HMD makes the OST-HMD unable to completely blocklight from the external environment, so that the virtual drawn contentcannot be independently presented on the display. In the MR applicationscene, the optical design will lead to the so-called color-blendingproblem, that is, since the virtual display content and the color of thereal background environment are blended, the virtual content becomesdifficult to see clearly, especially when there is a low color contrastbetween the virtual content and the background environment.

Under such a background, it has become a practical demand to enhance thediscrimination between the virtual display content and the realbackground environment. However, there is no universal solution suitablefor the mainstream commercial OST-HMD at present, and there are manypossibilities to be explored in this direction. In the relatedstrategies to relieve the color-blending problem on the OST-HMD,controlling the opaqueness of a display pixel by increasing anadditional hardware system has been proved to be an effective hardwaresolution. In such a solution, complete or partial blocking effectbetween the virtual display content and the real background environmentmay be achieved by adjusting the opaqueness of the display pixel, sothat the discrimination between the virtual display content and the realbackground environment is improved. However, this hardware solutionusually does not have scalability and is difficult to realize theminiaturization of equipment overall dimension, and the production costis high.

In addition to the hardware solution, some software solutions performaccurate pixel-level matching on the display picture and the backgroundenvironment to subtract the color component of the real backgroundenvironment from the virtual display content, so that the realbackground environment corresponding to the mixed virtual displaycontent is almost invisible and the discrimination between the displaypicture and the background environment is finally improved. However, thesolution for the mainstream commercial OST-HMD is adopted in the dailyenvironment to reduce the brightness of the virtual display content, sothat the transparency of the virtual display content is increased, andthe discrimination between the virtual display content and the realbackground environment is reduced.

However, there is also a software solution for the mainstream commercialOST-HMD to increase the opaqueness of the virtual display content byenhancing the brightness of the virtual display content, so that thediscrimination between the virtual display content and the realbackground environment is improved. However, this solution will reducethe contrast of the virtual display content itself and lose the specificdetails, so this solution is not suitable for the virtual content withcomplex texture and has low universality.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a color contrastenhanced rendering method, device and system suitable for an opticalsee-through head-mounted display. According to the color contrastenhanced rendering method, an optimal rendering color based on anoriginal rendering color and a background environment color is foundpixel by pixel under the condition of meeting a series of constraintconditions.

The technical solution of the present invention is as follows:

A color contrast enhanced rendering method suitable for an opticalsee-through head-mounted display comprises the following steps:

-   -   (1) acquiring a background environment in real time to obtain a        background video and pre-processing the background video to        obtain a processed background video;    -   (2) converting an original rendering color and a processed        background video color from their own color spaces to a CIELAB        color space;    -   (3) finding an optimal rendering color based on the original        rendering color and the processed video color in the CIELAB        space according to a set color difference constraint, a        chromaticity saturation constraint, a brightness constraint and        a just/minimum noticeable difference constraint; and    -   (4) after converting the optimal rendering color from the CIELAB        color space back to the original rendering color space,        replacing the original rendering color with the converted        optimal rendering color for real-time rendering.

The original rendering color refers to an original color displayed bythe OST-HMD on its display screen before this technical solution isapplied; the original rendering color space refers to a color spaceadopted by the original rendering color; the background video colorrefers to a color of a video of a background environment behind theOST-HMD acquired by video acquisition equipment (such as a camera); thebackground video color space refers to a color space adopted by thebackground video color; and converting the original rendering color andthe processed background video color from their own color spaces to theCIELAB color space refers to that the original rendering color isconverted from the original rendering color space to the CIELAB colorspace and the background video color is converted from the video colorspace to the CIELAB color space. The original rendering color space maybe a color space such as RGB, HSV, YUV, CIEXYZ and CIELAB, and the videocolor space may be a color space such as RGB, HSV, YUV and YCbCr.

Preferably, the step of pre-processing the background video comprises:

performing Gaussian blur processing on the background video,specifically, filtering the background video by using a Gaussian filtercore to obtain a blur effect, wherein the mathematical description ofthe Gaussian filter core is as follows:

${G\left( {x,y} \right)} = {\frac{1}{2\pi\sigma^{2}}e^{- \frac{x^{2} + y^{2}}{2\sigma^{2}}}}$

wherein x and y respectively represent horizontal and vertical distancesfrom the pixel to a center pixel, and σ represents a standard deviationof the selected Gaussian distribution;

performing visual field correction on the background video subjected toGaussian processing to realize pixel accuracy one-to-one mapping of aframe image of the background video and a rendering picture, wherein acorrection formula is:

$\left\{ \begin{matrix}{u = {{s_{u}x} + b_{u}}} \\{v = {{s_{v}y} + b_{v}}}\end{matrix} \right.$

wherein u and v respectively represent texture coordinates of therendering picture in horizontal and vertical directions, x and yrespectively represent texture coordinates of the frame image of thebackground video in the horizontal and vertical directions, s_(u) ands_(v) respectively represent scaling coefficients of the texturecoordinates of the frame image of the background video in the horizontaland vertical directions, and b_(u) and b_(v) respectively representoffsets of the texture coordinates of the frame image of the backgroundvideo in the horizontal and vertical directions.

The step (2) specifically includes:

(2-1) converting the original rendering color and the processedbackground video color from their own original color spaces to theCIELAB color space; and

(2-2) scaling the three-dimensional coordinates of the originalrendering color and the background video color in the CIELAB color spaceto a unit sphere range, that is, an Euclidean distance from thethree-dimensional coordinates corresponding to any color in the CIELABcolor space after scaling to the origin is less than or equal to 1.

In the step (3),

the color difference constraint means that a color difference of theoptimal rendering color and the original rendering color should be keptwithin a certain range, and the color different constraint is defined asfollows:ΔE* _(ab)(l _(opt) ,l _(d))≤λ_(E)

wherein ΔE*_(ab)(•) represents the color difference of the two colors inthe CIELAB color space, l_(opt) and l_(d) respectively represent theoptimal rendering color and the original rendering color, and λ_(E)represents a color difference threshold with a positive value;

the chromaticity saturation constraint means that a chromaticitysaturation of the optimal rendering color should not be reduced, and thechromaticity saturation constraint is defined as follows:ch _(opt) −ch _(d)≥0

wherein ch_(opt) and ch_(d) respectively represent chromaticitysaturations of the optimal rendering color and the original renderingcolor in the CIELAB color space;

the brightness constraint means that a brightness difference of theoptimal rendering color and the original rendering color should be keptwithin a certain range, and the brightness constraint is defined asfollows:ΔL*(l _(opt) ,l _(d))≤λ_(L)

wherein ΔL*(•) represents the brightness difference of the two colors inthe CIELAB color space, and λ_(L) represents a brightness threshold witha positive value; and

the just noticeable difference constraint means that the optimalrendering color and the background video color should have a colordifference distinguishable by human eyes, and the just noticeabledifference constraint is defined as follows:ΔE* _(ab)(l _(opt) ,l _(d))≥λ_(JND)

wherein l_(b) represents the background video color, and λ_(JND)represents a just noticeable difference with a positive value.

The step (3) step specifically comprises:

(3-1) calculating an ideal optimal rendering color by using thefollowing formula:

$I = {{- \frac{B}{dis{t\left( {B,O} \right)}}} = {{- n}or{m\left( \overset{\rightarrow}{OB} \right)}}}$

wherein I and B respectively represent three-dimensional coordinates ofthe ideal optimal rendering color and the background video colorsubjected to Gaussian blur in the CIELAB color space which is scaled toa unit sphere, O represents an origin of coordinates and, O is also acenter of the unit sphere, dist(•) represents an Euclidean distancebetween two points in the space, norm(•) represents that a vector isnormalized, {right arrow over (OB)} represents a vector with a startingpoint at O and an end point at B; and

(3-2) applying the color difference constraint to the ideal optimalrendering color, wherein the method for applying the color differenceconstraint is as follows:{right arrow over (DE)}=min(dist(D,I),λ′_(E))·norm({right arrow over(DI)})

wherein D and E respectively represent three-dimensional coordinates ofthe original rendering color and the ideal optimal rendering color towhich the color difference constraint is applied in the CIELAB colorrange scaled to the unit sphere, min(•) represents taking a smallervalue of the two three-dimensional coordinates, λ′_(E) represents acolor different threshold scaled to the unit sphere range, and {rightarrow over (DI)} represents a three-dimensional vector with a startingpoint at D and an end point at I;

(3-3) continuing to apply the chromaticity saturation constraint to theideal optimal rendering color to which the color difference constraintis applied, and making a superscript ′ represent taking a projectionvector of a vector on a a*Ob* plane in the CIELAB color space, whereinthe method for applying the chromaticity saturation constraint is asfollows:

${\overset{\rightarrow}{D⁢C} = {{t_{ch} \cdot {\overset{\rightarrow}{DE}}_{ch}^{\prime}} + {\overset{\rightarrow}{DE}}_{h}^{\prime}}}{t_{ch} = \left\{ \begin{matrix}{1,{{{if}\theta_{ch}} \leq {90{^\circ}}}} \\{0,{{{if}\theta_{ch}} > {90{^\circ}}}}\end{matrix} \right.}$

wherein {right arrow over (DC)} represents a {right arrow over (DE)}′vector after the chromaticity saturation constraint is applied, DE′_(ch)represents a vector component of a {right arrow over (DE)}′ vector in a{right arrow over (OD)}′ vector direction, that is, a variation vectorof the chromaticity saturation constraint, {right arrow over (DE)}′_(h)represents another vector component of the {right arrow over (DE)}′ in adirection vertical to the {right arrow over (DE)}′_(ch), vector, thatis, {right arrow over (DE)}′_(h)={right arrow over (DE)}′−{right arrowover (DE)}′_(ch), and θ_(ch) represents an angle from the {right arrowover (OD)}′ vector to the {right arrow over (DE)}′ vector;

(3-4) continuing to apply the brightness constraint to the ideal optimalrendering color to which the color different constraint and thechromaticity saturation constraint are applied, wherein the method forapplying the brightness constraint is as follows:{right arrow over (DL)}=(1−|cos θ_(l)|)·{right arrow over (DE)} _(L*)

wherein {right arrow over (DE)}_(L*) represents a vector component of a{right arrow over (DE)} vector on a L* axis in the CIELAB color spacescaled to the unit sphere, {right arrow over (DL)} represents a {rightarrow over (DE)}_(L*) vector after the brightness constraint is applied,θ_(l) represents an angle from a positive direction of the L* axis tothe {right arrow over (DE)} vector; and

(3-5) continuing to apply the just noticeable difference constraint tothe ideal optimal rendering color to which the color differentconstraint, the chromaticity saturation constraint and the brightnessconstraint are applied, wherein the method for applying the justnoticeable difference constraint is as follows:{right arrow over (DP)}={right arrow over (DC)}+{right arrow over (DL)}L _(opt) =D+r·{right arrow over (DP)}

wherein L_(opt) represents three-dimensional coordinates of the optimalrendering color in the CIELAB color space scaled to the unit sphere, andr represents a scaling coefficient: when an Euclidean distance from P toB is less than the just noticeable difference scaled to the unit sphererange, r is less than 1, so as to reduce a module length of a {rightarrow over (DP)} vector with a starting point at D and an end point atP, so that an Euclidean distance from the end point of the reduced{right arrow over (DP)} vector to B is not less than the just noticeabledifference scaled to the unit sphere range, and if an Euclidean distancefrom the P to the B is already greater than or equal to the justnoticeable difference scaled to the unit sphere unit, r is equal to 1.

The step (4) specifically comprises:

(4-1) scaling the optimal rendering color in the CIELAB color spacescaled to the unit sphere range back to a coordinate range of theoriginal CIELAB color space;

(4-2) converting the optimal rendering color in the CIELAB color spaceback to the original rendering color space; and

(4-3) replacing the original rendering color with the converted optimalrendering color for real-time rendering.

In the present invention, the background environment may be acquired inreal time by a video camera of the OST-HMD, only the color differencethreshold λ_(E) and the just noticeable difference λ_(JND) need to bedetermined by a user, and other parameters such as the brightnessthreshold λ_(L) are automatically set by the present invention duringoperation.

A color contrast enhanced rendering device suitable for an opticalsee-through head-mounted display comprises:

an acquisition and processing module, configured to acquire a backgroundenvironment in real time to obtain a background video and performGaussian blur and visual field correction on the background video toobtain a processed background video;

a color space conversion module, configured to convert the originalrendering color and the processed background video color from their ownoriginal color spaces to a CIELAB color space, and further configured toconvert the optimal rendering color from the CIELAB color space to theoriginal rendering color space;

an optimal rendering color calculation module, configured to find anoptimal rendering color based on the original rendering color and theprocessed video color in the CIELAB space according to a set colordifference constraint, a chromaticity saturation constraint, abrightness constraint and a just noticeable difference constraint; and

a rendering module, configured to replace the original rendering colorby the converted optimal rendering color for real-time rendering.

A color contrast enhanced rendering system suitable for an opticalsee-through head-mounted display processor comprises a memory and acomputer program which is stored in the processor and capable of beingexecuted by the processor, wherein the computer program, when beingexecuted by the processor, implements the above color contrast enhancedrendering method suitable for the optical see-through head-mounteddisplay.

Compared with the prior art, the present invention has the followingbeneficial effects:

the application range is wide and is not limited to a certain specificapplication program or certain specific hardware; the rendering methodof the present invention is a full-screen post-processing effect, may beapplied to rendering frameworks such as Vulkan, DirectX and OpenGL andother commercial game engines, and may also be popularized to variousdifferent OST-HMDs;

within the color difference threshold given by a user, by increasing thecolor contrast between the virtual display content and the realbackground environment, the discrimination between the virtual displaycontent and the real background environment is improved on the premiseof ensuring the presentation of the virtual content as much as possible,and the visibility reduction or detail loss of the virtual displaycontent caused by the prior art is avoided; and

the rendering method supports various mixed reality application sceneswithout any pre-preparation process and may be applied to various commonbackground environments.

BRIEF DESCRIPTIONS OF THE DRAWINGS

In order to describe the technical solutions in the embodiments of thepresent invention or in the prior art more clearly, the followingbriefly introduces the accompanying drawings required for describing theembodiments or the prior art. Apparently, the accompanying drawings inthe following description show merely some embodiments of the presentinvention, and a person of ordinary skill in the art may also deriveother drawings from these accompanying drawings without creativeefforts.

FIG. 1 is a flowchart of a color contrast enhanced drawing methodsuitable for an optical see-through head-mounted display according to anembodiment of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS

In order to make the objectives, technical schemes and advantages of thepresent invention more clearly, the present invention will be furtherdescribed below in detail with reference to the accompanying drawingsand embodiments. It should be understood that the specific embodimentsdescribed herein are only used to explain the present invention and arenot used to limit the protection scope of the present invention.

Referring to FIG. 1 , a color contrast enhanced rendering methodsuitable for an optical see-through head-mounted display in thisembodiment comprises the following steps:

S01: a background environment is acquired in real time by a video cameraof the optical see-through head-mounted display to obtain a backgroundvideo.

In this embodiment, a resolution ratio of the generated background videowas 768×432, and a frame rate is 24 frames/second.

S02: the background video was filtered by using a Gaussian filter coreto obtain a blur effect.

In this embodiment, the mathematical description of the Gaussian filtercore required to perform Gaussian filtering is as follows:

${G\left( {x,y} \right)} = {\frac{1}{2\pi\sigma^{2}}e^{- \frac{x^{2} + y^{2}}{2\sigma^{2}}}}$

wherein x and y respectively represent horizontal and vertical distancesfrom the pixel to a center pixel, and σ represents a standard deviationof the selected Gaussian distribution. In this embodiment, a standarddeviation of the selected Gaussian distribution was 3.

S03: the background video was subjected to visual field correction byusing a visual field correction formula.

In this embodiment, the visual field correction was defined as follows:

$\left\{ \begin{matrix}{u = {{s_{u}x} + b_{u}}} \\{v = {{s_{v}y} + b_{v}}}\end{matrix} \right.$

wherein u and v respectively represent texture coordinates of therendering picture in horizontal and vertical directions, x and yrespectively represent texture coordinates of the frame image of thebackground video in the horizontal and vertical directions, s_(u) ands_(v) respectively represent scaling coefficients of the texturecoordinates of the frame image of the background video in the horizontaland vertical directions, and b_(u) and b_(v) respectively representoffsets of the texture coordinates of the frame image of the backgroundvideo in the horizontal and vertical directions. In this embodiment, thevalues of both s_(u) and s_(v) were 0.65, and the values of b_(u) andb_(v) were 0.13 and 0.17 respectively.

S04: the original rendering color and the background video color wereconverted from an RGB color space to an CIELAB color space pixel bypixel, the three-dimensional coordinates of the original rendering colorand the background video color in the CIELAB color space were scaled toa unit sphere range, that is, an Euclidean distance from thethree-dimensional coordinates corresponding to any color in the CIELABcolor space after scaling to an origin was less than or equal to 1.

S05: according to a color different threshold given by a user, anoptimal rendering color based on the original display color and thebackground video color was found pixel by pixel under the condition ofmeeting a series of constraint conditions.

In this embodiment, there were four constraint conditions, which wererespectively defined as follows:

(a) a color difference constraint means that a color difference of theoptimal rendering color and the original rendering color should be keptwithin a certain range, and the color different constraint was definedas follows:ΔE* _(ab)(l _(opt) ,l _(d))≤λ_(E)

wherein ΔE*_(ab)(•) represents the color difference of the two colors inthe CIELAB color space, l_(opt) and l_(d) respectively represent theoptimal rendering color and the original rendering color, and λ_(E)represents a color difference threshold with a positive value;

(b) a chromaticity saturation constraint means that a chromaticitysaturation of the optimal rendering color should not be reduced, and thechromaticity saturation constraint was defined as follows:ch _(opt) −ch _(d)≥0

wherein ch_(opt) and ch_(d) respectively represent chromaticitysaturations of the optimal rendering color and the original renderingcolor in the CIELAB color space;

(c) a brightness constraint means that a brightness difference of theoptimal rendering color and the original rendering color should be keptwithin a certain range, and the brightness constraint was defined asfollows:ΔL*(l _(opt) ,l _(d))≤λ_(L)

wherein ΔL*(•) represents the brightness difference of the two colors inthe CIELAB color space, and λ_(L) represents a brightness threshold witha positive value; and

(d) a just noticeable difference constraint means that the optimalrendering color and the background video color should have a colordifference distinguishable by human eyes, and the just noticeabledifference constraint was defined as follows:ΔE* _(ab)(l _(opt) ,l _(b))≥λ_(JND)

wherein l_(b) represents the background video color, and λ_(JND)represents a just noticeable difference with a positive value.

The S05 specifically comprises:

S05-1: an ideal optimal rendering color was calculated. According tothis embodiment, the ideal optimal rendering color was calculated byusing the following formula:

$I = {{- \frac{B}{dis{t\left( {B,O} \right)}}} = {- {{norm}\left( \overset{\rightarrow}{OB} \right)}}}$

wherein I and B respectively represent three-dimensional coordinates ofthe ideal optimal rendering color and the background video colorsubjected to Gaussian blur in the CIELAB color space which was scaled toa unit sphere, O represents an origin of coordinates and, O is also acenter of the unit sphere, dist(•) represents an Euclidean distancebetween two points in the space, norm(•) represents that a vector isnormalized, {right arrow over (OB)} represents a vector with a startingpoint at O and an end point at B; and

S05-2: the color difference constraint was applied to the ideal optimalrendering color. The method for applying the color difference constraintwas as follows:{right arrow over (DE)}=min(dist(D,I),λ′_(E))·norm({right arrow over(DI)})

wherein D and E respectively represent three-dimensional coordinates ofthe original rendering color and the ideal optimal rendering color towhich the color difference constraint was applied in the CIELAB colorrange scaled to the unit sphere, min(•) represents taking a smallervalue of the two three-dimensional coordinates, λ′_(E) represents acolor different threshold scaled to the unit sphere range, and {rightarrow over (DI)} represents a three-dimensional vector with a startingpoint at D and an end point at I;

S05-3: the chromaticity saturation constraint was continuously appliedto the ideal optimal rendering color to which the color differenceconstraint was applied. A superscript ′ is made to represent taking aprojection vector of a vector on a a*Ob* plane in the CIELAB colorspace. The method for applying the chromaticity saturation constraintwas as follows:

${\overset{\rightarrow}{D⁢C} = {{t_{ch} \cdot {\overset{\rightarrow}{DE}}_{ch}^{\prime}} + {\overset{\rightarrow}{DE}}_{h}^{\prime}}}{t_{ch} = \left\{ \begin{matrix}{1,{{{if}\theta_{ch}} \leq {90{^\circ}}}} \\{0,{{{if}\theta_{ch}} > {90{^\circ}}}}\end{matrix} \right.}$

wherein {right arrow over (DC)} represents a {right arrow over (DE)}′vector after the chromaticity saturation constraint was applied, {rightarrow over (DE)}′_(ch) represents a vector component of a {right arrowover (DE)}′ vector in a {right arrow over (OD)}′ vector direction, thatis, a variation vector of the chromaticity saturation constraint, {rightarrow over (DE)}′_(h) represents another vector component of the {rightarrow over (DE)}′ in a direction vertical to the {right arrow over(DE)}′_(ch) vector, that is, DE′_(h)={right arrow over (DE)}′−{rightarrow over (DE)}′_(ch), and θ_(ch) represents an angle from the {rightarrow over (OD)}′ vector to the {right arrow over (DE)}′ vector;

S05-4: the brightness constraint was continuously applied to the idealoptimal rendering color to which the color different constraint and thechromaticity saturation constraint were applied. The method for applyingthe brightness constraint was as follows:{right arrow over (DL)}=(1−|cos θ_(l)|)·{right arrow over (DE)} _(L*)

wherein {right arrow over (DE)}_(L*) represents a vector component of a{right arrow over (DE)} vector on a L* axis in the CIELAB color spacescaled to the unit sphere, {right arrow over (DL)} represents a {rightarrow over (DE)}_(L*) vector after the brightness constraint wasapplied, θ_(l) represents an angle from a positive direction of the L*axis to the {right arrow over (DE)} vector.

S05-5: the just noticeable difference constraint was continuouslyapplied to the ideal optimal rendering color to which the colordifferent constraint, the chromaticity saturation constraint and thebrightness constraint were applied. In this embodiment, the method forapplying the just noticeable difference constraint was as follows:{right arrow over (DP)}={right arrow over (DC)}+{right arrow over (DL)}L _(opt) =D+r·{right arrow over (DP)}

wherein L_(opt) represents three-dimensional coordinates of the optimalrendering color in the CIELAB color space scaled to the unit sphere, andr represents a scaling coefficient:

when an Euclidean distance from P to B was less than the just noticeabledifference scaled to the unit sphere range, r was less than 1, so that amodule length of a {right arrow over (DP)} vector with a starting pointat D and an end point at P was reduced, and an Euclidean distance fromthe end point of the reduced {right arrow over (DP)} vector to B was notless than the just noticeable difference scaled to the unit sphererange, and if an Euclidean distance from the P to the B was alreadygreater than or equal to the just noticeable difference scaled to theunit sphere unit, r was equal to 1. In this embodiment, the value of thejust noticeable difference before scaling was 2.3.

S06: the optimal rendering color was converted from the CIELAB colorspace to a CIE XYZ color space pixel by pixel, and the optimal renderingcolor in the CIE XYZ color space was converted back to the RGB colorspace.

S07: the original rendering color was replaced by the optimal renderingcolor in the RGB color space for real-time rendering.

The experimental simulation result was as shown in Table 1 when thevirtual display content was drawn by the method of this embodiment,wherein λ′_(E) represents a color difference threshold given by a user.It may be seen that by the method of the present invention, a colorcontrast between a rendering color and a background color may beenhanced, so that a discrimination between the virtual content and thereal background environment was obviously improved.

TABLE 1 Color difference threshold λ′_(Ε) = 0.3 λ′_(Ε) = 0.5 λ′_(Ε) =0.8 Area ratio of pixels with enhanced color 73.83% 87.62% 92.72%contrast to rendering pixels in full screen

An embodiment further provides a color contrast enhanced renderingdevice suitable for an optical see-through head-mounted display,including:

-   -   an acquisition and processing module, configured to acquire a        background environment in real time to obtain a background video        and perform Gaussian blur and visual field correction on the        background video to obtain a processed background video;    -   a color space conversion module, configured to convert the        original rendering color and the processed background video        color from an RGB color space to a CIELAB color space, and        further configured to convert the optimal rendering color from        the CIELAB color space to the RGB color space;    -   an optimal rendering color calculation module, configured to        find an optimal rendering color based on the original rendering        color and the processed video color in the CIELAB space        according to a set color difference constraint, a chromaticity        saturation constraint, a brightness constraint and a just        noticeable difference constraint; and    -   a rendering module, configured to replace the original rendering        color by the optimal rendering color in the RGB color space for        real-time rendering.

It should be noted that when the color contrast enhanced renderingdevice suitable for the optical see-through head-mounted displayprovided by the above embodiment performs color contrast enhancedrendering, division of the foregoing function modules should be used asan example for description, and the foregoing functions may be allocatedto and completed by different function modules as required. That is, aninternal structure of a server was divided into different functionmodules, to complete all or some of the functions described above. Thatis, an internal structure of a terminal or a server was divided intodifferent function modules to complete all or some of the functionsdescribed above. In addition, the color contrast enhanced renderingdevice suitable for the optical see-through head-mounted displayprovided by the above embodiment belongs to the same concept as thecolor contrast enhanced rendering method suitable for the opticalsee-through head-mounted display provided by the above embodiment. Thedetails of the specific implementation process of the color contrastenhanced rendering device suitable for the optical see-throughhead-mounted display were shown in the embodiment of the color contrastenhanced rendering method suitable for the optical see-throughhead-mounted display, which are not elaborated herein.

An embodiment further provides a color contrast enhanced renderingsystem suitable for an optical see-through head-mounted display,including a processor, a memory and a computer program which was storedin the processor and capable of being executed by the processor, whereinthe computer program, when being executed by the processor, implementsthe above color contrast enhanced rendering method suitable for theoptical see-through head-mounted display.

The memory may include one or more computer readable storage mediumsthat may be non-transitory. The memory may further include a high-speedrandom access memory and a nonvolatile memory, such as one or more diskstorage devices and flash storage devices. In some embodiments, thenon-transitory computer readable storage mediums in the memory were usedfor storing at least one instruction which is used for being executed bythe processor to implement the color contrast enhanced rendering methodsuitable for the optical see-through head-mounted display provided bythe method embodiments of the present application.

The above specific embodiments have described the technical solutionsand beneficial effects of the present invention in detail. It should beunderstood that the above is only the most preferred embodiment of thepresent invention and is not used to limit the present invention. Anymodification, supplement and equivalent substitution made within theprinciple scope of the present invention should be included in theprotection scope of the present invention.

What is claimed is:
 1. A color contrast enhanced rendering methodsuitable for an optical see-through head-mounted display, the colorcontrast enhanced rendering method comprising the following steps: (1)acquiring a background environment in real time to obtain a backgroundvideo and pre-processing the background video to obtain a processedbackground video; (2) converting an original rendering color and aprocessed background video color to a CIELAB color space; (3)calculating an ideal optimal rendering color by calculating,${I = {{- \frac{B}{dis{t\left( {B,O} \right)}}} = {- {{norm}\left( \overset{\rightarrow}{OB} \right)}}}},$ in the CIELAB color space, and then sequentially applying a colordifference constraint, a chromaticity saturation constraint, abrightness constraint and a just noticeable difference constraint to theideal optimal rendering color to obtain an optimal rendering color,wherein I and B respectively represent three-dimensional coordinates ofthe ideal optimal rendering color and the background video colorsubjected to Gaussian blur in the CIELAB color space which is scaled toa unit sphere, O represents an origin of coordinates and, O is also acenter of the unit sphere, dist(•) represents an Euclidean distancebetween two points in the space, norm(•) represents that a vector isnormalized, {right arrow over (OB)} represents a vector with a startingpoint at O and an end point at B; and (4) after converting the optimalrendering color from the CIELAB color space back to the originalrendering color space, replacing the original rendering color with theconverted optimal rendering color for real-time rendering.
 2. The colorcontrast enhanced rendering method suitable for the optical see-throughhead-mounted display according to claim 1, wherein the step ofpre-processing the background video comprises: performing Gaussian blurprocessing on the background video; and performing visual fieldcorrection on the background video subjected to Gaussian processing torealize pixel accuracy one-to-one mapping of a frame image of thebackground video and a rendering picture, a correction formula being:$\left\{ \begin{matrix}{u = {{s_{u}x} + b_{u}}} \\{v = {{s_{v}y} + b_{v}}}\end{matrix} \right.$ wherein u and v respectively represent texturecoordinates of the rendering picture in horizontal and verticaldirections, x and y respectively represent texture coordinates of theframe image of the background video in the horizontal and verticaldirections, s_(u) and s_(v) respectively represent scaling coefficientsof the texture coordinates of the frame image of the background video inthe horizontal and vertical directions, and b_(u) and b_(v) respectivelyrepresent offsets of the texture coordinates of the frame image of thebackground video in the horizontal and vertical directions.
 3. The colorcontrast enhanced rendering method suitable for the optical see-throughhead-mounted display according to claim 1, wherein the step (2)specifically comprises: (2-1) converting the original rendering colorand the processed background video color from their own original colorspaces to the CIELAB color space; and (2-2) scaling thethree-dimensional coordinates of the original rendering color and thebackground video color in the CIELAB color space to a unit sphere range,that is, an Euclidean distance from the three-dimensional coordinatescorresponding to any color in the CIELAB color space after scaling tothe origin being less than or equal to
 1. 4. The color contrast enhancedrendering method suitable for the optical see-through head-mounteddisplay according to claim 1, wherein in the step (3), the colordifference constraint means that a color difference of the optimalrendering color and the original rendering color should be kept within acertain range, and the color different constraint is defined as follows:ΔE* _(ab)(l _(opt) ,l _(d))≤λ_(E) wherein Δ_(E)*_(ab)(•) represents thecolor difference of the two colors in the CIELAB color space, l_(opt)and l_(d) respectively represent the optimal rendering color and theoriginal rendering color, and λ_(E) represent a color differencethreshold with a positive value; the chromaticity saturation constraintmeans that a chromaticity saturation of the optimal rendering colorshould not be reduced, and the chromaticity saturation constraint isdefined as follows:ch _(opt) −ch _(d)≥0 wherein ch_(opt) and ch_(d) respectively representchromaticity saturations of the optimal rendering color and the originalrendering color in the CIELAB color space; the brightness constraintmeans that a brightness difference of the optimal rendering color andthe original rendering color should be kept within a certain range, andthe brightness constraint is defined as follows:ΔL*(l _(opt) ,l _(d))≤λ_(L) wherein ΔL* (•) represents the brightnessdifference of the two colors in the CIELAB color space, and λ_(L)represents a brightness threshold with a positive value; and the justnoticeable difference constraint means that the optimal rendering colorand the background video color should have a color differencedistinguishable by human eyes, and the just noticeable differenceconstraint is defined as follows:ΔE* _(a b)(l _(opt) ,l _(b))≤λ_(JND) wherein l_(b) represents thebackground video color, and Δ_(JND) represents a just noticeabledifference with a positive value.
 5. The color contrast enhancedrendering method suitable for the optical see-through head-mounteddisplay according to claim 1, wherein the step of sequentially applyinga color difference constraint, a chromaticity saturation constraint, abrightness constraint and a just noticeable difference constraint to theideal optimal rendering color to obtain an optimal rendering colorcomprises: applying the color difference constraint to the ideal optimalrendering color, the method for applying the color difference constraintbeing as follows:{right arrow over (DE)}=min(dist(D,I),λ′_(E))·norm({right arrow over(DI)}) wherein D and E respectively represent three-dimensionalcoordinates of the original rendering color and the ideal optimalrendering color to which the color difference constraint is applied inthe CIELAB color range scaled to the unit sphere, min(•) representstaking a smaller value of the two three-dimensional coordinates, λ′_(E)represents a color different threshold scaled to the unit sphere range,and {right arrow over (DI)} represents a three-dimensional vector with astarting point at D and an end point at I; continuing to apply thechromaticity saturation constraint to the ideal optimal rendering colorto which the color difference constraint is applied, and making asuperscript ′ represent taking a projection vector of a vector on aa*Ob* plane in the CIELAB color space, the method for applying thechromaticity saturation constraint being as follows:${\overset{\rightarrow}{D⁢C} = {{t_{ch} \cdot {\overset{\rightarrow}{DE}}_{ch}^{\prime}} + {\overset{\rightarrow}{DE}}_{h}^{\prime}}}{t_{ch} = \left\{ \begin{matrix}{1,{{{if}\theta_{ch}} \leq {90{^\circ}}}} \\{0,{{{if}\theta_{ch}} > {90{^\circ}}}}\end{matrix} \right.}$ wherein {right arrow over (DC)} represents a{right arrow over (DE)}′ vector after the chromaticity saturationconstraint is applied, {right arrow over (DE)}′_(ch) represents a vectorcomponent of a {right arrow over (DE)}′ vector in a {right arrow over(OD)}′ vector direction, that is, a variation vector of the chromaticitysaturation constraint, {right arrow over (DE)}′_(h) represents anothervector component of the {right arrow over (DE)}′ in a direction verticalto the {right arrow over (DE)}′_(ch), vector, that is, {right arrow over(DE)}′_(h)={right arrow over (DE)}′−{right arrow over (DE)}′_(ch), andθ_(ch) represents an angle from the {right arrow over (OD)}′ vector tothe {right arrow over (DE)}′ vector; continuing to apply the brightnessconstraint to the ideal optimal rendering color to which the colordifferent constraint and the chromaticity saturation constraint wereapplied, the method for applying the brightness constraint being asfollows:{right arrow over (DL)}=(1−|cos θ_(l)|)·{right arrow over (DE)} _(L*)wherein {right arrow over (DE)}_(L) represents a vector component of a{right arrow over (DE)} vector on a L* axis in the CIELAB color spacescaled to the unit sphere, {right arrow over (DL)} represents a {rightarrow over (DE)}_(L*) vector after the brightness constraint is applied,θ_(l) represents an angle from a positive direction of the L* axis tothe {right arrow over (DE)} vector; and continuing to apply the justnoticeable difference constraint to the ideal optimal rendering color towhich the color different constraint, the chromaticity saturationconstraint and the brightness constraint are applied, the method forapplying the just noticeable difference constraint being as follows:{right arrow over (DP)}={right arrow over (DC)}+{right arrow over (DL)}L _(opt) =D+r·{right arrow over (DP)} wherein L_(opt) representsthree-dimensional coordinates of the optimal rendering color in theCIELAB color space scaled to the unit sphere, and r represents a scalingcoefficient: when an Euclidean distance from P to B is less than thejust noticeable difference scaled to the unit sphere range, r is lessthan 1, so as to reduce a module length of a {right arrow over (DP)}vector with a starting point at D and an end point at P, so that anEuclidean distance from the end point of the reduced {right arrow over(DP)} vector to B is not less than the just noticeable difference scaledto the unit sphere range, and if an Euclidean distance from the P to theB is already greater than or equal to the just noticeable differencescaled to the unit sphere unit, r is equal to
 1. 6. The color contrastenhanced rendering method suitable for the optical see-throughhead-mounted display according to claim 1, wherein the step (4)specifically comprises: (4-1) scaling the optimal rendering color in theCIELAB color space scaled to the unit sphere range back to a coordinaterange of the original CIELAB color space; (4-2) converting the optimalrendering color in the CIELAB color space back to the original renderingcolor space; and (4-3) replacing the original rendering color with theconverted optimal rendering color for real-time rendering.
 7. A colorcontrast enhanced rendering device suitable for an optical see-throughhead-mounted display, comprising: an acquisition and processing moduleconfigured to acquire a background environment in real time to obtain abackground video and perform Gaussian blur and visual field correctionon the background video to obtain a processed background video; a colorspace conversion module configured to convert an original renderingcolor and a processed background video color to a CIELAB color space andfurther configured to convert the optimal rendering color from theCIELAB color space to the original rendering color space; an optimalrendering color calculation module configured to calculate an idealoptimal rendering color by calculating,${I = {{- \frac{B}{dis{t\left( {B,O} \right)}}} = {- {{norm}\left( \overset{\rightarrow}{OB} \right)}}}},$ in the CIELAB color space and then sequentially apply a colordifference constraint, a chromaticity saturation constraint, abrightness constraint and a just noticeable difference constraint to theideal optimal rendering color to obtain an optimal rendering color,wherein I and B respectively represent three-dimensional coordinates ofthe ideal optimal rendering color and the background video colorsubjected to Gaussian blur in the CIELAB color space which is scaled toa unit sphere, O represents an origin of coordinates and, O is also acenter of the unit sphere, dist(•) represents an Euclidean distancebetween two points in the space, norm(•) represents that a vector isnormalized, {right arrow over (OB)} represents a vector with a startingpoint at O and an end point at B; and a rendering module configured toreplace the original rendering color with the converted optimalrendering color for real-time rendering.
 8. A color contrast enhancedrendering system suitable for an optical see-through head-mounteddisplay, comprising a processor, a memory and a computer program storedin the processor and capable of being executed by the processor, whereinthe computer program, when executed by the processor, implements thecolor contrast enhanced rendering method suitable for the opticalsee-through head-mounted display as defined in claim 1.